Supporting Attention Allocation in Multitask Environments : Effects of Likelihood Alarm Systems on Trust, Behavior, and Performance

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Objective: The aim of the current study was to investigate potential benefits of likelihood alarm systems (LASs) over binary alarm systems (BASs) in a multitask environment. Background: Several problems are associated with the use of BASs, because most of them generate high numbers of false alarms. Operators lose trust in the systems and ignore alarms or cross-check all of them when other information is available. The first behavior harms safety, whereas the latter one reduces productivity. LASs represent an alternative, which is supposed to improve operators’ attention allocation. Method: We investigated LASs and BASs in a dual-task paradigm with and without the possibility to cross-check alerts with raw data information. Participants’ trust in the system, their behavior, and their performance in the alert and the concurrent task were assessed. Results: Reported trust, compliance with alarms, and performance in the alert and the concurrent task were higher for the LAS than for the BAS. The cross-check option led to an increase in alert task performance for both systems and a decrease in concurrent task performance for the BAS, which did not occur in the LAS condition. Conclusion: LASs improve participants’ attention allocation between two different tasks and therefore lead to an increase in alert task and concurrent task performance. The performance maximum is achieved when LAS is combined with a cross-check option for validating alerts with additional information. Application: The use of LASs instead of BASs in safety-related multitask environments has the potential to increase safety and productivity likewise

Asymmetric effects of false positive and false negative indications on the verification of alerts in different risk conditions

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Indications from alerts or alarm systems can be the trigger for decisions, or they can elicit further information search. We report an experiment on the tendency to collect additional information after receiving system indications. We varied the proclivity of the alarm system towards false positive or false negative indications and the perceived risk of the situation. Results showed that false alarm-prone systems led to more frequent re-checking following both alarms and non-alarms in the high risk condition, whereas miss-prone systems led to high re-checking rates only for non-alarms, representing an asymmetry effect. Increasing the risk led to more re-checks with all alarm systems, but it had a stronger impact in the false alarm-prone condition. Results regarding the relation of risk and the asymmetry effect of false negative and false positive indications are discussed

Is Operators' Compliance with Alarm Systems a Product of Rational Consideration?

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Most theories about operators' responses to alarm systems suggest that the operators' behavior is guided by their trust towards the system which in turn results from the subjective perception of system properties, namely the perceived reliability of the alarm system. However, some doubts about that assumption have arisen as recent research has not proven the mediating effect of trust. The purpose of this research was to examine the relationship between alarm system properties, trust, and behavior. The alarm reliability was varied while keeping the other system properties constant. It was found that participants' response-rates to alarms were predicted by their perceived alarm reliabilities. However, no mediation by trust could be established. These results suggest that operators' behavior is not always guided by their trust towards the system. Under specific circumstances their compliance rather depends on rational consideration regarding the most efficient strategy

Measures of Reliance and Compliance in Aided Visual Scanning

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Objective: We study the dependence or independence of reliance and compliance as two responses to alarms to understand the mechanisms behind these responses. Background: Alarms, alerts, and other binary cues affect user behavior in complex ways. The suggestion has been made that there are two different responses to alerts—compliance (the tendency to perform an action cued by the alert) and reliance (the tendency to refrain from actions as long as no alert is issued). The study tests the degree to which these two responses are indeed independent. Method: An experiment tested the effects of the positive and negative predictive values of the alerts (PPV and NPV) on measures of compliance and reliance based on cutoff settings, response times, and subjective confidence. Results: For cutoff settings and response times, compliance was unaffected by the irrelevant NPV, whereas reliance depended on the irrelevant PPV. For subjective estimates, there were no significant effects of the irrelevant variables. Conclusion: Results suggest that compliance is relatively stable and unaffected by irrelevant information (the NPV), whereas reliance is also affected by the PPV. The results support the notion that reliance and compliance are separate, but related, forms of trust. Application: False alarm rates, which affect PPV, determine both the response to alerts (compliance) and the tendency to limit precautions when no alert is issued (reliance)

On the Relation Between Reliance and Compliance in an Aided Visual Scanning Task

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Alarms, alerts, and other binary cues affect user behavior in complex ways. One relevant distinction is the suggestion that there are two different responses to alerts – compliance (the tendency to perform an action cued by the alert) and reliance (the tendency to refrain from actions as long as no alert is issued). An experiment tested the dependence of the two behaviors on the Positive and Negative Predictive Values of the alerts (PPV and NPV) to determine whether these are indeed two different behaviors. Results suggest that the compliance is relatively stable and unaffected by irrelevant information (the NPV), while reliance is also affected by the PPV. The results are discussed in terms of multiple-process theories of trust in information sources

Decision-making and response strategies in interaction with alarms: the impact of alarm reliability, availability of alarm validity information and workload

Responding to alarm systems which usually commit a number of false alarms and/or misses involves decision-making under uncertainty. Four laboratory experiments including a total of n=256 participants were conducted to gain comprehensive insight into humans’ dealing with this uncertainty. Specifically, it was investigated how responses to alarms/nonalarms are affected by the predictive validities of these events, and to what extent response strategies depend on whether or not the validity of alarms/nonalarms can be cross-checked against other data. Among others, the results suggest that, without cross-check possibility (experiment 1), low levels of predictive validity of alarms (≤0.5) led most participants to use one of two different strategies which both involved non-responding to a significant number of alarms (cry-wolf effect). Yet, providing access to alarm validity information reduced this effect dramatically (experiment 2). This latter result emerged independent of the effort needed for cross-checkings of alarms (experiment 3), but was affected by the workload imposed by concurrent tasks (experiment 4). Theoretical and practical consequences of these results for decision-making and response selection in interaction with alarm systems, as well as the design of effective alarm systems are discussed

Benefits of Decision-Support by Likelihood versus Binary Alarm Systems : Does the number of stages make a difference?

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Recent research has shown that the use of 3-stage likelihood alarm systems (LAS) has the potential to mitigate performance deficits associated with the use of binary alarm systems (BAS). The additional likelihood information can guide operators’ behavior and improve their decision-making accuracy. Comparisons of LAS with different numbers of stages are missing so far. Therefore, the current study compared a BAS with a 3-stage LAS and a 4-stage LAS. Participants were found to make significantly fewer wrong decisions with the 4-stage LAS than with the other two systems, and still significantly fewer errors with the 3-stage LAS compared to the BAS. We found that this performance benefit resulted from a reduced number of false alarms, whereas no difference was found with regard to misses. Results are further discussed with regard to their theoretical implications for LAS and threshold setting in BAS

Investigating benefits of likelihood alarm systems in presence of alarm validity information

Dieser Beitrag ist mit Zustimmung des Rechteinhabers aufgrund einer (DFG geförderten) Allianz- bzw. Nationallizenz frei zugänglich.This publication is with permission of the rights owner freely accessible due to an Alliance licence and a national licence (funded by the DFG, German Research Foundation) respectively.Providing operators additional information helping them to validate alarms has been found to be a countermeasure for problems related to the cry wolf effect (i.e., operators ignoring alarms). Adding information can be realized with likelihood alarm systems (LAS) or with access to alarm validity information (AVI). The two studies presented here examined behavior and performance consequences of the combination of LAS and AVI in multi-task settings. It was investigated to what extent concurrent task performance and alert task performance depend on characteristics of the LAS (i.e. proportion of different alert types) and cost of cross-checking AVI. Results suggest that those LAS characteristics varied here do not influence participants’ performance. Secondly, no benefit of LAS over binary alarm systems (BAS) emerged when increasing the cost of accessing AVI. Results are further discussed with regard to participants’ response patterns

The Impact of Visual and Cognitive Dual-Task Demands on Traffic Perception During Road Crossing of Older and Younger Pedestrians

With the help of the current experiment, we wanted to learn more about the impact of visually demanding vs. cognitively demanding secondary tasks on the attention allocation of older pedestrians during the phase of traffic perception within the process of road crossing. For this purpose, we used two different road crossing tasks as well as two different secondary tasks. The road crossing “stop task” was a signal detection task, where an approaching car had to be detected. The road crossing “go task” was a dynamic visual search task, where the resolution of a busy road situation had to be identified. The visual secondary task was a static visual search task and the cognitive secondary task was a 1-back (memory) task. One younger group (≤ 30 years) and one older group (≥ 65 years) of participants completed the tasks as single vs. dual-tasks in all possible combinations. Performance was measured through errors and response time; in addition, the subjective workload was assessed via NASA-TLX. Analyses show that the visual secondary task reduces performance in the road crossing more strongly than the cognitive task, while the visual task itself is less impaired by the road crossing tasks than is the cognitive task. Overall, performance diminishes from single to dual-task completion. Results further indicate age effects in terms of increased errors and response time for older compared to younger participants. In addition to these age effects, age-specific dual-task effects emerge for response time in the go task along with the visual task as well as for response time in the cognitive task along with the go task. Subjective workload is higher in the dual-task conditions than in the single tasks. Findings are discussed with regard to theoretical and practical implications

Effects of Trust, Self-Confidence, and Feedback on the Use of Decision Automation

Operators often fail to rely sufficiently on alarm systems. This results in a joint human-machine (JHM) sensitivity below the one of the alarm system. The ‘confidence vs. trust hypothesis’ assumes the use of the system depends on the weighting of both values. In case of higher confidence, the task is performed manually, if trust is higher, the user relies on the system. Thus, insufficient reliance may be due to operators’ overconfidence in their own abilities and/or insufficient trust in the decision automation, but could be mitigated by providing feedback. That was investigated within a signal detection task, supported by a system with either higher sensitivity (HSS) or lower sensitivity (LSS) than the human, while being provided with feedback or not. We expected disuse of the LSS and insufficiently reliance on the HSS, in the condition without feedback. The feedback was expected to increase reliance on the HSS through an increase in trust and/or decreases in confidence, and thus, improve performance. Hypotheses were partly supported. Confidence in manual performance was similar to trust in the HSS even though humans’ sensitivity was significantly lower than systems’ sensitivity. While confidence had not effect on reliance or JHM sensitivity, trust was found to be positively related with both. We found disuse of the HSS, that could be improved through feedback, increasing also trust and JHM sensitivity. However, contrary to ‘confidence vs. trust’ expectations, participants were also found to make use of the LSS. This misuse could not be reduced by feedback. Results indicate the use of feedback being beneficial for the overall performance (with HSS only). Findings do not support the idea that misuse or disuse of the system may result from comparison of confidence and trust. We suppose it may rather be the product of users’ wrong strategy of function allocation, based on the underlying idea of team work in combination with missing assignment of responsibility. We discuss this alternative explanation